Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JSON Module rewrite with custom JSONPath implementation #974

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open

JSON Module rewrite with custom JSONPath implementation #974

wants to merge 14 commits into from

Conversation

Vijay-Nirmal
Copy link
Contributor

@Vijay-Nirmal Vijay-Nirmal commented Jan 29, 2025 •
edited
Loading

As part of this PR, the JSON module implementation has been rewritten using a custom implementation of JsonPath to achieve better performance and compatibility with Redis. As part of this rewrite other parts of the Json module have also been rewritten

Main Changes:

  • Custom implementation of JSONPath evaluator, which is a port of Newtonsoft.Json with many optimizations and some bug fixes
  • Implemented full support for JSON.SET command but there is a known issue which should prevent this PR from merging
  • Implemented full support for JSON.GET command but there is a limitation in STJ that doesn't allow us to customize the output format using custom characters. Most likely we can never support this feature in Garnet if we use STJ [API Proposal]: Allow inheritance for Utf8JsonWriter class dotnet/runtime#111899
  • Added separate benchmark for JsonPath evaluator and Json comments with a big JSON
  • Removed JsonPath.Net package reference

Miscellaneous:

  • Made CmdStrings class public in CmdStrings.cs to allow external access.
  • Changed ExistOptions enum to public in RespEnums.cs for broader accessibility.

Known Issues:

  • If value that needs to be updated is null using JSON.SET command then the update will fail, it can't find the parent object to update the value. This is a limitation of the custom JSONPath implementation because of the way the Null is represented in JsonNode. Will find a way to support this in future without affecting the performance much. This issue shouldn't block the PR from being merged as this scenario was failing in the current implementation as well.
  • Utf8JsonWriter doesn't allow enough customizability to support INDENT, NEWLINE and SPACE options in JSON.GET command. We can't inherit the Utf8JsonWriter class as well as it's a sealed class to write our own logic to customize it. This feature will not be supported in STJ in future as well [API Proposal]: Allow inheritance for Utf8JsonWriter class dotnet/runtime#111899 (comment). We have only two options, either Garnet can never support the customizability provided by Redis for certain JSON commands, or Garnet should use Newtonsoft.Json (which supports customisation). Both of these options are not ideal.

TODO

  • Initial setup
  • Adding more test cases
  • Code cleanup and code documentation
  • Creating a separate document page for JSON module
  • And other things I forgot ;-)

Custom JSONPath Benchmark

Method JsonPath Mean Error Ratio Allocated Alloc Ratio
Garnet $..[?(@.price < 10)] 916.7 ns 2.31 ns baseline 1168 B ****
'JsonPath.Net (json-everything)' $..[?(@.price < 10)] 10,000.2 ns 96.90 ns 10.91x slower 25112 B 21.50x more
Newtonsoft.Json $..[?(@.price < 10)] 2,797.5 ns 21.01 ns 3.05x slower 7456 B 6.38x more
JsonCraft.JsonPath $..[?(@.price < 10)] 1,261.5 ns 5.05 ns 1.38x slower 1288 B 1.10x more
BlushingPenguin.JsonPath $..[?(@.price < 10)] 3,799.7 ns 32.10 ns 4.14x slower 9304 B 7.97x more
Hyperbee.Json $..[?(@.price < 10)] 9,814.6 ns 67.13 ns 10.71x slower 11176 B 9.57x more
JsonCons.JsonPath $..[?(@.price < 10)] 3,425.9 ns 40.47 ns 3.74x slower 7392 B 6.33x more
Garnet $..['bicycle','price'] 504.6 ns 4.45 ns baseline 1096 B ****
'JsonPath.Net (json-everything)' $..['bicycle','price'] 6,467.7 ns 61.68 ns 12.82x slower 13568 B 12.38x more
Newtonsoft.Json $..['bicycle','price'] 739.3 ns 2.73 ns 1.47x slower 696 B 1.57x less
JsonCraft.JsonPath $..['bicycle','price'] 724.2 ns 5.52 ns 1.44x slower 1504 B 1.37x more
BlushingPenguin.JsonPath $..['bicycle','price'] 1,982.2 ns 9.10 ns 3.93x slower 3736 B 3.41x more
Hyperbee.Json $..['bicycle','price'] 3,123.5 ns 9.64 ns 6.19x slower 2264 B 2.07x more
JsonCons.JsonPath $..['bicycle','price'] 1,381.0 ns 11.70 ns 2.74x slower 3784 B 3.45x more
Garnet $..* 432.3 ns 2.63 ns baseline 752 B ****
'JsonPath.Net (json-everything)' $..* 8,651.7 ns 59.45 ns 20.01x slower 16352 B 21.74x more
Newtonsoft.Json $..* 649.2 ns 3.79 ns 1.50x slower 320 B 2.35x less
JsonCraft.JsonPath $..* 434.6 ns 2.62 ns 1.01x slower 696 B 1.08x less
BlushingPenguin.JsonPath $..* 1,902.5 ns 14.87 ns 4.40x slower 3344 B 4.45x more
Hyperbee.Json $..* 4,946.5 ns 15.62 ns 11.44x slower 3856 B 5.13x more
JsonCons.JsonPath $..* 1,297.5 ns 9.11 ns 3.00x slower 4200 B 5.59x more
Garnet $..author 369.6 ns 3.34 ns baseline 768 B ****
'JsonPath.Net (json-everything)' $..author 5,706.1 ns 51.25 ns 15.44x slower 11784 B 15.34x more
Newtonsoft.Json $..author 577.4 ns 2.00 ns 1.56x slower 336 B 2.29x less
JsonCraft.JsonPath $..author 540.2 ns 7.39 ns 1.46x slower 696 B 1.10x less
BlushingPenguin.JsonPath $..author 1,763.5 ns 8.95 ns 4.77x slower 3360 B 4.38x more
Hyperbee.Json $..author 2,264.7 ns 14.35 ns 6.13x slower 2056 B 2.68x more
JsonCons.JsonPath $..author 1,053.2 ns 6.47 ns 2.85x slower 2560 B 3.33x more
Garnet $..book[0,1] 452.3 ns 1.78 ns baseline 968 B ****
'JsonPath.Net (json-everything)' $..book[0,1] 5,997.6 ns 25.62 ns 13.26x slower 12776 B 13.20x more
Newtonsoft.Json $..book[0,1] 650.3 ns 2.11 ns 1.44x slower 584 B 1.66x less
JsonCraft.JsonPath $..book[0,1] 651.0 ns 2.70 ns 1.44x slower 920 B 1.05x less
BlushingPenguin.JsonPath $..book[0,1] 2,001.7 ns 12.00 ns 4.43x slower 3616 B 3.74x more
Hyperbee.Json $..book[0,1] 2,539.5 ns 9.99 ns 5.61x slower 2056 B 2.12x more
JsonCons.JsonPath $..book[0,1] 1,364.7 ns 9.12 ns 3.02x slower 3168 B 3.27x more
Garnet $.store..price 458.9 ns 4.75 ns baseline 984 B ****
'JsonPath.Net (json-everything)' $.store..price 6,008.5 ns 36.83 ns 13.10x slower 12360 B 12.56x more
Newtonsoft.Json $.store..price 586.5 ns 3.21 ns 1.28x slower 472 B 2.08x less
JsonCraft.JsonPath $.store..price 611.1 ns 4.81 ns 1.33x slower 952 B 1.03x less
BlushingPenguin.JsonPath $.store..price 1,472.7 ns 8.97 ns 3.21x slower 3320 B 3.37x more
Hyperbee.Json $.store..price 1,983.6 ns 8.77 ns 4.32x slower 1792 B 1.82x more
JsonCons.JsonPath $.store..price 1,074.7 ns 5.82 ns 2.34x slower 2472 B 2.51x more
Garnet $.store.* 106.0 ns 1.20 ns baseline 312 B ****
'JsonPath.Net (json-everything)' $.store.* 744.4 ns 5.11 ns 7.02x slower 2552 B 8.18x more
Newtonsoft.Json $.store.* 174.1 ns 1.58 ns 1.64x slower 568 B 1.82x more
JsonCraft.JsonPath $.store.* 119.5 ns 1.49 ns 1.13x slower 384 B 1.23x more
BlushingPenguin.JsonPath $.store.* 149.0 ns 1.48 ns 1.41x slower 512 B 1.64x more
Hyperbee.Json $.store.* 762.2 ns 4.13 ns 7.19x slower 1440 B 4.62x more
JsonCons.JsonPath $.store.* 335.8 ns 3.44 ns 3.17x slower 1224 B 3.92x more
Garnet $.store.bicycle.color 141.6 ns 1.07 ns baseline 408 B ****
'JsonPath.Net (json-everything)' $.store.bicycle.color 1,056.1 ns 9.62 ns 7.46x slower 3184 B 7.80x more
Newtonsoft.Json $.store.bicycle.color 196.3 ns 1.94 ns 1.39x slower 632 B 1.55x more
JsonCraft.JsonPath $.store.bicycle.color 171.4 ns 1.42 ns 1.21x slower 384 B 1.06x less
BlushingPenguin.JsonPath $.store.bicycle.color 224.1 ns 1.95 ns 1.58x slower 688 B 1.69x more
Hyperbee.Json $.store.bicycle.color 303.4 ns 1.24 ns 2.14x slower 208 B 1.96x less
JsonCons.JsonPath $.store.bicycle.color 369.9 ns 4.87 ns 2.61x slower 1184 B 2.90x more
Garnet $.store.book[-1:] 155.8 ns 1.37 ns baseline 464 B ****
'JsonPath.Net (json-everything)' $.store.book[-1:] 1,089.0 ns 9.82 ns 6.99x slower 3184 B 6.86x more
Newtonsoft.Json $.store.book[-1:] 203.0 ns 2.66 ns 1.30x slower 656 B 1.41x more
JsonCraft.JsonPath $.store.book[-1:] 181.8 ns 1.35 ns 1.17x slower 472 B 1.02x more
BlushingPenguin.JsonPath $.store.book[-1:] 231.2 ns 1.62 ns 1.48x slower 696 B 1.50x more
Hyperbee.Json $.store.book[-1:] 734.4 ns 5.58 ns 4.71x slower 1232 B 2.66x more
JsonCons.JsonPath $.store.book[-1:] 528.3 ns 5.05 ns 3.39x slower 1480 B 3.19x more
Garnet $.store.book[:2] 159.3 ns 0.97 ns baseline 464 B ****
'JsonPath.Net (json-everything)' $.store.book[:2] 1,365.1 ns 17.31 ns 8.57x slower 3496 B 7.53x more
Newtonsoft.Json $.store.book[:2] 211.0 ns 1.91 ns 1.32x slower 648 B 1.40x more
JsonCraft.JsonPath $.store.book[:2] 183.3 ns 2.00 ns 1.15x slower 472 B 1.02x more
BlushingPenguin.JsonPath $.store.book[:2] 248.2 ns 3.32 ns 1.56x slower 688 B 1.48x more
Hyperbee.Json $.store.book[:2] 858.6 ns 5.51 ns 5.39x slower 1232 B 2.66x more
JsonCons.JsonPath $.store.book[:2] 513.4 ns 2.72 ns 3.22x slower 1504 B 3.24x more
Garnet $.store.book[?(@.author && @.title)] 386.8 ns 4.28 ns baseline 1080 B ****
'JsonPath.Net (json-everything)' $.store.book[?(@.author && @.title)] 3,065.6 ns 12.00 ns 7.93x slower 8936 B 8.27x more
Newtonsoft.Json $.store.book[?(@.author && @.title)] 552.9 ns 5.34 ns 1.43x slower 1752 B 1.62x more
JsonCraft.JsonPath $.store.book[?(@.author && @.title)] 476.1 ns 4.31 ns 1.23x slower 1088 B 1.01x more
BlushingPenguin.JsonPath $.store.book[?(@.author && @.title)] 632.3 ns 6.43 ns 1.63x slower 1896 B 1.76x more
Hyperbee.Json $.store.book[?(@.author && @.title)] 2,149.2 ns 13.36 ns 5.56x slower 2856 B 2.64x more
JsonCons.JsonPath $.store.book[?(@.author && @.title)] 1,427.3 ns 7.46 ns 3.69x slower 2944 B 2.73x more
Garnet *$.store.book[?(@.author =~ /.Waugh/)] 745.1 ns 4.08 ns baseline 1128 B ****
'JsonPath.Net (json-everything)' $.store.book[?(@.author =~ /.*Waugh/)] NA NA ? NA ?
Newtonsoft.Json $.store.book[?(@.author =~ /.*Waugh/)] 761.5 ns 4.13 ns 1.02x slower 1456 B 1.29x more
JsonCraft.JsonPath $.store.book[?(@.author =~ /.*Waugh/)] 1,031.2 ns 3.33 ns 1.38x slower 1296 B 1.15x more
BlushingPenguin.JsonPath $.store.book[?(@.author =~ /.*Waugh/)] 1,228.5 ns 5.53 ns 1.65x slower 1912 B 1.70x more
Hyperbee.Json $.store.book[?(@.author =~ /.*Waugh/)] NA NA ? NA ?
JsonCons.JsonPath $.store.book[?(@.author =~ /.*Waugh/)] 2,277.5 ns 15.96 ns 3.06x slower 5224 B 4.63x more
Garnet $.store.book[?(@.category == 'fiction')] 623.0 ns 2.29 ns baseline 1280 B ****
'JsonPath.Net (json-everything)' $.store.book[?(@.category == 'fiction')] 2,842.9 ns 24.55 ns 4.56x slower 7256 B 5.67x more
Newtonsoft.Json $.store.book[?(@.category == 'fiction')] 479.3 ns 7.47 ns 1.30x faster 1480 B 1.16x more
JsonCraft.JsonPath $.store.book[?(@.category == 'fiction')] 638.1 ns 4.19 ns 1.02x slower 1144 B 1.12x less
BlushingPenguin.JsonPath $.store.book[?(@.category == 'fiction')] 980.0 ns 5.23 ns 1.57x slower 2120 B 1.66x more
Hyperbee.Json $.store.book[?(@.category == 'fiction')] 1,800.4 ns 5.69 ns 2.89x slower 2584 B 2.02x more
JsonCons.JsonPath $.store.book[?(@.category == 'fiction')] 1,139.8 ns 6.19 ns 1.83x slower 2560 B 2.00x more
Garnet $.store.book[?(@.price < 10)].title 525.0 ns 2.91 ns baseline 976 B ****
'JsonPath.Net (json-everything)' $.store.book[?(@.price < 10)].title 3,236.1 ns 25.51 ns 6.16x slower 8040 B 8.24x more
Newtonsoft.Json $.store.book[?(@.price < 10)].title 532.9 ns 4.36 ns 1.02x slower 1632 B 1.67x more
JsonCraft.JsonPath $.store.book[?(@.price < 10)].title 836.6 ns 4.01 ns 1.59x slower 1136 B 1.16x more
BlushingPenguin.JsonPath $.store.book[?(@.price < 10)].title 958.4 ns 8.36 ns 1.83x slower 1912 B 1.96x more
Hyperbee.Json $.store.book[?(@.price < 10)].title 2,092.5 ns 22.10 ns 3.99x slower 2592 B 2.66x more
JsonCons.JsonPath $.store.book[?(@.price < 10)].title 1,505.9 ns 11.04 ns 2.87x slower 2824 B 2.89x more
Garnet $.store.book[?(@.price > 10 && @.price < 20)] 675.9 ns 2.62 ns baseline 1256 B ****
'JsonPath.Net (json-everything)' $.store.book[?(@.price > 10 && @.price < 20)] 4,510.4 ns 61.00 ns 6.67x slower 11056 B 8.80x more
Newtonsoft.Json $.store.book[?(@.price > 10 && @.price < 20)] 675.4 ns 5.78 ns 1.00x faster 2232 B 1.78x more
JsonCraft.JsonPath $.store.book[?(@.price > 10 && @.price < 20)] 1,104.8 ns 4.48 ns 1.63x slower 1528 B 1.22x more
BlushingPenguin.JsonPath $.store.book[?(@.price > 10 && @.price < 20)] 1,377.9 ns 6.62 ns 2.04x slower 2680 B 2.13x more
Hyperbee.Json $.store.book[?(@.price > 10 && @.price < 20)] 2,898.3 ns 13.43 ns 4.29x slower 3384 B 2.69x more
JsonCons.JsonPath $.store.book[?(@.price > 10 && @.price < 20)] 2,336.2 ns 14.53 ns 3.46x slower 3832 B 3.05x more
Garnet $.store.book[*] 141.4 ns 0.63 ns baseline 352 B ****
'JsonPath.Net (json-everything)' $.store.book[*] 1,335.6 ns 15.02 ns 9.45x slower 3472 B 9.86x more
Newtonsoft.Json $.store.book[*] 199.6 ns 1.60 ns 1.41x slower 632 B 1.80x more
JsonCraft.JsonPath $.store.book[*] 160.2 ns 1.23 ns 1.13x slower 376 B 1.07x more
BlushingPenguin.JsonPath $.store.book[*] 220.0 ns 2.18 ns 1.56x slower 648 B 1.84x more
Hyperbee.Json $.store.book[*] 804.0 ns 4.48 ns 5.69x slower 1320 B 3.75x more
JsonCons.JsonPath $.store.book[*] 371.1 ns 2.77 ns 2.63x slower 1248 B 3.55x more
Garnet $.store.book[*].author 204.4 ns 1.78 ns baseline 496 B ****
'JsonPath.Net (json-everything)' $.store.book[*].author 2,073.9 ns 22.38 ns 10.15x slower 4992 B 10.06x more
Newtonsoft.Json $.store.book[*].author 284.0 ns 2.62 ns 1.39x slower 784 B 1.58x more
JsonCraft.JsonPath $.store.book[*].author 322.7 ns 3.31 ns 1.58x slower 520 B 1.05x more
BlushingPenguin.JsonPath $.store.book[*].author 337.0 ns 2.70 ns 1.65x slower 816 B 1.65x more
Hyperbee.Json $.store.book[*].author 1,231.4 ns 3.88 ns 6.03x slower 1528 B 3.08x more
JsonCons.JsonPath $.store.book[*].author 501.9 ns 5.90 ns 2.46x slower 1384 B 2.79x more
Garnet $.store.book[0].title 178.1 ns 1.54 ns baseline 448 B ****
'JsonPath.Net (json-everything)' $.store.book[0].title 1,496.1 ns 11.22 ns 8.40x slower 4088 B 9.12x more
Newtonsoft.Json $.store.book[0].title 256.2 ns 3.25 ns 1.44x slower 760 B 1.70x more
JsonCraft.JsonPath $.store.book[0].title 231.8 ns 1.17 ns 1.30x slower 440 B 1.02x less
BlushingPenguin.JsonPath $.store.book[0].title 308.6 ns 1.80 ns 1.73x slower 832 B 1.86x more
Hyperbee.Json $.store.book[0].title 368.7 ns 0.85 ns 2.07x slower 208 B 2.15x less
JsonCons.JsonPath $.store.book[0].title 469.6 ns 4.86 ns 2.64x slower 1264 B 2.82x more

Note and a disclaimer: JsonCraft.JsonPath is a package I published. If you need Garent's implementation as a separate package you can use JsonCraft.JsonPath package. In the benchmark, it looks slightly slower because it's benchmarked against the JsonElement implementation instead of JsonNode. You can find the exact same implementations as Garent in Experimental folder

Garnet BDN Benchmark

In the main branch as of 762a9d7

Method Params Mean Error StdDev Gen0 Allocated
ModuleJsonGetCommand None 122.330 us 1.5715 us 1.4700 us - 72804 B
ModuleJsonSetCommand None 206.487 us 1.9531 us 1.8270 us - 223200 B
ModuleJsonGetDeepPath None 260.595 us 2.3140 us 2.0513 us 0.4883 368808 B
ModuleJsonGetArrayPath None 352.331 us 1.8106 us 1.6936 us - 61600 B
ModuleJsonGetArrayElementsPath None 6.870 us 0.0438 us 0.0410 us - 800 B
ModuleJsonGetFilterPath None 391.566 us 2.9899 us 2.4967 us - 108800 B
ModuleJsonGetRecursive None 15,777.044 us 273.2510 us 255.5992 us 31.2500 27778471 B

In this PR as of 7650f5f: (More improvements to come)

Method Params Mean Error StdDev Gen0 Allocated
ModuleJsonGetCommand None 117.554 us 1.3885 us 1.2988 us - 77604 B
ModuleJsonSetCommand None 120.205 us 1.2426 us 1.1623 us - 59200 B
ModuleJsonGetDeepPath None 128.231 us 2.0790 us 1.9447 us - 96804 B
ModuleJsonGetArrayPath None 342.911 us 2.0679 us 1.8331 us - 58400 B
ModuleJsonGetArrayElementsPath None 6.895 us 0.0315 us 0.0279 us - 800 B
ModuleJsonGetFilterPath None 347.648 us 3.5888 us 3.3570 us - 69600 B
ModuleJsonGetRecursive None 5,764.681 us 78.0601 us 73.0175 us 7.8125 4612193 B

Out of scope of this PR

Some of these items I will be raising future PRs to address it

  1. 1st known issue - Issue with updating null value
  2. Modify regex to follow Redis JsonPath Regex pattern/syntax instead of the standard /
  3. Adding other commands in the JSON Module
  4. Return parsing errors instead of throwing in JsonPath implementation
  5. Matching JsonPath parsing error with Redis error message
  6. Experimenting with custom JsonPath to be created using Span
  7. Make the RegexMatchTimeout configurable

badrishc
badrishc reviewed Jan 29, 2025
View reviewed changes
playground/GarnetJSON/GarnetJSON.csproj
</ItemGroup>

<ItemGroup>
<ProjectReference Include="..\..\libs\server\Garnet.server.csproj" />
<InternalsVisibleTo Include="Garnet.test" Key="0024000004800000940000000602000000240000525341310004000001000100011b1661238d3d3c76232193c8aa2de8c05b8930d6dfe8cd88797a8f5624fdf14a1643141f31da05c0f67961b0e3a64c7120001d2f8579f01ac788b0ff545790d44854abe02f42bfe36a056166a75c6a694db8c5b6609cff8a2dbb429855a1d9f79d4d8ec3e145c74bfdd903274b7344beea93eab86b422652f8dd8eecf530d2" />
</ItemGroup>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of playground, can we use a top level folder called modules and put this in there?

@Vijay-Nirmal Vijay-Nirmal marked this pull request as ready for review February 1, 2025 20:23
@Vijay-Nirmal Vijay-Nirmal requested a review from badrishc February 1, 2025 20:23
@Vijay-Nirmal
Copy link
Contributor Author

Vijay-Nirmal commented Feb 1, 2025 •
edited
Loading

I am not sure why the code style check is failing, imports are already in order. It's not because of the licence header because there are other files with a licence header that are not part of the error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@darrenge darrenge Awaiting requested review from darrenge darrenge is a code owner

@badrishc badrishc Awaiting requested review from badrishc

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Vijay-Nirmal @badrishc